Multi-camera vehicular vision system

ABSTRACT

A vehicular vision system includes forward viewing camera and a rear camera each having a respective field of view exterior of the vehicle. Each of the cameras has an LVDS chip and connects with a control via a respective mono coaxial cable. Each mono coaxial cable carries via LVDS image data captured by the respective camera. Image data captured by the forward viewing camera and carried to the control by the respective mono coaxial cable is processed at the control for object detection. The control generates an output provided to a video display device of the vehicle, the video display device having a video display screen viewable by a driver of the vehicle. The video display screen displays video images of an area rearward of the vehicle derived from image data captured by the rear camera and carried to the control by the respective mono coaxial cable.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/125,903, filed Sep. 10, 2018, now U.S. Pat. No. 10,640,040,which is a continuation of U.S. patent application Ser. No. 14/359,341,filed May 20, 2014, now U.S. Pat. No. 10,071,687, which is a 371national phase filing of PCT Application No. PCT/US2012/066571, filedNov. 27, 2012, which claims the filing benefit of U.S. provisionalapplications, Ser. No. 61/650,667, filed May 23, 2012; Ser. No.61/605,409, filed Mar. 1, 2012; Ser. No. 61/579,682, filed Dec. 23,2011; Ser. No. 61/565,713, filed Dec. 1, 2011; and Ser. No. 61/563,965,filed Nov. 28, 2011, which are hereby incorporated herein by referencein their entireties, and U.S. patent application Ser. No. 14/359,341 isa continuation-in-part of PCT/US2012/066570, filed Nov. 27, 2012, whichclaims the filing benefit of U.S. provisional applications, Ser. No.61/605,409, filed Mar. 1, 2012, and Ser. No. 61/563,965, filed Nov. 28,2011.

FIELD OF THE INVENTION

The present invention relates to imaging systems or vision systems forvehicles.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties.

SUMMARY OF THE INVENTION

The present invention provides a vision system or imaging system for avehicle that utilizes one or more cameras to capture images exterior ofthe vehicle, and provides the communication/data signals, includingcamera data or image data that is processed and, responsive to suchimage processing, detects an object at or near the vehicle and in thepath of travel of the vehicle, such as when the vehicle is backing up.The present invention transfers the intelligence from the camera to animage displaying device or image display or cluster, central display orhead unit (later referred as head unit or HU) or to a mobile devicewired or wireless connected or attached to—or plugged into the head unit(as an app). The data transfer rate can be enhanced by LVDS having rawdata transmitted as described in U.S. Pat. No. 7,979,536, which ishereby incorporated herein by reference in its entirety. Optionally, theusers may be served with dump rear cameras, with DAS software functions,running independent and remote from the camera, but not in anothercontrol device. Thus, a business model may be provided that sells anapp, not the hardware, to the end users or consumers.

According to an aspect of the present invention, a vision system for avehicle includes at least one camera or image sensor disposed at avehicle and having a field of view exterior of the vehicle, and adisplay device operable to display images for viewing by a driver of thevehicle. The camera provides almost raw image data to a display deviceand has a control channel for a data line, and wherein a graphic engineor image processing runs as a routine at the display device.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a vision system and imagingsensors or cameras that provide exterior fields of view in accordancewith the present invention;

FIG. 2 is a schematic of an automotive vision camera for providing dataand getting controlled from an image processing control device, with theimage processing control device providing image data to a head unit orother display device;

FIG. 3A is a schematic of an automotive vision camera with graphicprocessing board incorporated within the same device, whereby theprocessed image data gets provided to a head unit or other displaydevice;

FIG. 3B is a schematic of an automotive vision camera with graphicprocessing board incorporated within the same device, whereby theprocessed image data gets provided to a head unit or other displaydevice and with the camera being controlled via a back channel;

FIG. 3C is a schematic of an automotive vision camera with graphicprocessing board incorporated within the same device, whereby theprocessed image data gets provided to a head unit or other displaydevice and with an additional bidirectional channel for exchangingcontrol data or parameter;

FIG. 3D is a schematic of an example of a graphic processingarchitecture solution embedded into an automotive vision camera (systemon chip), whereby the processed image data is provided to a head unit orother display device by at least one channel but maybe two in parallelto different output bus interfaces for the camera features, where thebusses may be used mono-directional or semi-directional (beside RGB888and NTSC) for control, and shown with RMII, but RGMII may be usedalternatively;

FIG. 4 is a schematic of an automotive vision camera that providesmostly raw image data to the head unit, which is carrying out any kindof graphic processing, and there is no communication channel for datafrom the head unit to the camera, either directly or via gateways, inaccordance with the present invention;

FIG. 5A is a schematic of an automotive vision camera system of thepresent invention with graphic processing incorporated within the headunit by hardware and software, with raw image data (directly) providedto a head unit or other display device for further image processing, andwith an additional bidirectional channel for exchanging control data orparameter;

FIG. 5B is a schematic of an automotive vision camera system of thepresent invention with graphic processing incorporated within the headunit by hardware and software, with raw image data (directly) providedto a head unit or other display device for further image processing, andwith the camera being controlled via a back channel;

FIG. 6 is a schematic of an automotive vision camera system of thepresent invention with graphic processing incorporated within the headunit, and with raw image data provided to a head unit or other displaydevice for further image processing, and with the image data transferredvia different busses linked by a bus gateway, and with the camera beingcontrolled via a back channel (which may comprise one bidirectionalchannel);

FIG. 7 is a schematic of an automotive vision camera system of thepresent invention with graphic processing incorporated within the headunit, and with raw image data provided to a head unit or other displaydevice for further image processing, and with the image data transferredvia different busses linked by a bus gateway, and with the cameracontrol back channel established via bus gateway (which may comprise abidirectional channel on one or both bus systems);

FIG. 8 is a schematic of an automotive vision camera, which providesalmost raw data to the head unit or other display device and has acontrol channel which might be any kind of data line or bus, and withthe graphic engine (or the image processing) running as a routine on thehead unit or other display device, in accordance with the presentinvention;

FIG. 9 is a schematic of an automotive vision system according to FIG.8, which uses LVDS/Ethernet as an Image data transfer channel, inaccordance with the present invention;

FIG. 10 is a schematic of an automotive vision system according to FIGS.8 and 9, which uses LVDS/Ethernet as a bidirectional control channel andan image data transfer channel, in accordance with the presentinvention;

FIG. 11A is a schematic of an automotive vision multi cameraarchitecture, where all cameras are controlled by one control/image dataprocessing device, and where the image data processing device sendsprocessed image data to the head unit or other display device;

FIG. 11B is a schematic of an automotive vision multi cameraarchitecture, where all of the system cameras are controlled by onecontrol/image data processing device, and where the image dataprocessing device sends processed image data to the head unit or otherdisplay device, with an additional mono-directional or bidirectionalchannel for exchanging control data or parameter with the displaydevice;

FIG. 11C is a schematic of an automotive vision multi (smart) cameraarchitecture, where the cameras incorporate one part of the image dataprocessing chain and the control device processes a second oralternative part of the image data processing, and where the image dataprocessing device sends processed image data to the head unit or otherdisplay device;

FIG. 11D is a schematic of an automotive vision multi (smart) cameraarchitecture, where the cameras incorporate one part of the image dataprocessing chain and the control device processes a second oralternative part of the image data processing, and where the image dataprocessing device sends processed or raw image data to the head unit orother display device, which processes a third or alternative part of theimage data processing;

FIG. 11E is a schematic of an automotive vision multi (smart) cameraarchitecture, where the cameras incorporate one part of the image dataprocessing chain and the head unit or other display device processes asecond or alternative part of the image data processing, and where theimage data and controls from and to the cameras are collected and/ortransferred to the head unit or other display device by a router;

FIG. 11F is a schematic of an automotive vision multi cameraarchitecture where the cameras send almost raw data over a router to ahead unit or other display device via monodirectional data lines or buschannel, where the data may be compressed by the cameras or by therouter before sending;

FIG. 12A is a schematic of an automotive vision multi cameraarchitecture, where all of the cameras provide almost raw data to thehead unit or other display device direct without having a central imageprocessing device or router in between and have a control channel forexchanging control data or parameter with the head unit or displaydevice which might be any kind of data line or bus, and with the graphicengine (or the image processing) running on an integrated hardware onthe head unit or other display device, in accordance with the presentinvention;

FIG. 12B is a schematic of an automotive vision multi cameraarchitecture, where all of the cameras provide almost raw data to thehead unit or other display device direct without having a central imageprocessing device or router in between and have a monodirectional cameracontrol channel which may comprise any kind of data line or bus, andwith the graphic engine (or the image processing) running on anintegrated hardware on the head unit or other display device, inaccordance with the present invention;

FIG. 12C is a schematic of an automotive vision multi cameraarchitecture, where some or all of the cameras provide almost raw datato the head unit or other display device direct without having a centralimage processing or router device in between using monodirectional datachannels as like NTSC and with the graphic engine (or the imageprocessing) running on an integrated hardware on the head unit or otherdisplay device, in accordance with the present invention;

FIG. 13 is a schematic of an automotive vision multi cameraarchitecture, where all of the cameras providing almost raw data to thehead unit or other display device and with the graphic engine (or theimage processing) running as a routine mainly on the CPU, without havingdedicated vehicle camera image processing hardware on the head unit orother display device, in accordance with the present invention;

FIG. 14 is a schematic of an automotive vision system according to FIG.8, with the head unit or other display device conjuncted to a mobileinfotainment device or mobile phone or mobile device, and with thegraphic engine (or the image processing) running as an ‘app’(application) on the mobile device, in accordance with the presentinvention, and with the camera comprising a basic or baseline camera,which has no control input;

FIG. 15 is a schematic of an automotive vision system, with a singlecamera providing image date to the head unit or other display device,and with the camera controlled via a control channel, and with the headunit carrying communication hardware for communicating with the camera,and with the vision software or vision applications utilizing theincoming data for further (high level) processing, in accordance withthe present invention;

FIG. 16 is a schematic of an automotive vision system according to FIGS.8 and 14, with the head unit or other display device conjuncted to amobile infotainment device or mobile phone or mobile device, and withthe graphic engine (or the image processing) running as an ‘app’(application) on the mobile device, in accordance with the presentinvention, and with a single camera in use that is controlled via acontrol channel;

FIG. 17 is a schematic of an automotive vision system according to FIG.16, with the graphic engine (or the image processing) running as an‘app’ (application) on the mobile device, and with the softwareadditionally computing additional parameters and with the steering angletaken into account, which may be used for processing graphical steeringaid overlays to a rear camera image while backing up the vehicle, inaccordance with the present invention;

FIG. 18 is a schematic of an automotive vision system according to FIG.17, with the system processing image data captured by more than onecamera and by other driver assistant system sources such as ultrasoundsensors, Radar sensors, infrared and visual cameras, Lidar or Lasersensors, in accordance with the present invention, with the graphicengine (or the image processing) running as an ‘app’ (application) onthe mobile device;

FIG. 19A is a schematic of an automotive vision system according toFIGS. 8 to 18, with the system using apps certified by accordingcertification boards of governmental organs or mobile device companiesand/or OEMs, in accordance with the present invention;

FIG. 19B is a schematic of an automotive vision/driver assistant systemaccording to FIGS. 18 and 19A, with the system is connected to a mobilephone which transmits control commands to vehicle inherent devices (aslike warnings, invasive interaction);

FIG. 20 is a schematic of an automotive vision system having a busgateway within the path of image raw data between camera and mobileinfotainment device or mobile phone, with the back channel to the cameraestablished separately, and with the mobile infotainment device ormobile phone doing the graphic processing, in accordance with thepresent invention;

FIG. 21 is a schematic of an automotive vision system according to FIG.16, having non permanently mounted USB based cameras plugged into a USBport that is part of the vehicle's bus architecture that may have busgateways such as USB to CAN, with the Camera sending images via the USB,in accordance with the present invention;

FIG. 22 is a schematic of an automotive vision system having a busgateway within the path of image raw data between the camera and headunit or other display device, with the back channel to the cameraestablished separately, and with the head unit doing the graphicprocessing, in accordance with the present invention;

FIG. 23 is a schematic of an automotive vision system having a busgateway within the path of image raw data between camera and mobileinfotainment device or mobile phone, with the camera control backchannel established via the gateways (which may comprise a bidirectionalchannel on one or both bus systems), and with the mobile device doingthe graphic processing, in accordance with the present invention;

FIG. 24 is a schematic of an automotive vision system havingnon-permanently mounted USB based cameras plugged into any USB port thatis part of the vehicle's bus architecture, which might have bus gatewayssuch as USB to CAN, with the Camera sending images via the USB, and withthe camera control back channel established via the gateways (which maycomprise a bidirectional channel on one or both bus systems), and withthe mobile device doing the graphic processing, in accordance with thepresent invention;

FIG. 25 is a schematic of an automotive vision system having a busgateway within the path of image raw data between camera and head unit,with the camera control back channel established via the gateways (whichmay comprise a bidirectional channel on one or both bus systems), andwith the mobile device doing the graphic processing, in accordance withthe present invention;

FIG. 26A is a schematic of an automotive vision system with aconsecutive solution to FIG. 18, where the app visualizes driverassistant functions, and also controls outputs such as drivinginterventions or active warnings such as steering wheel or foot pedalvibrations, in accordance with the present invention;

FIG. 26B is a schematic of an automotive vision system with the systemusing apps certified by according certification boards of governmentalorgans or mobile device companies and/or OEMs, in accordance with thepresent invention;

FIG. 27 is a schematic of an automotive vision system having a busgateway within the path of image raw data between camera and head unit,with the back channel to the camera established separately, and with thehead unit or other display device doing the graphic processing, with thehead unit carrying communication hardware for communicating with thecamera, and with the vision software or vision applications utilizingthe incoming data for further (high level) processing, which may run ondesignated image processing hardware, in accordance with the presentinvention;

FIG. 28 is a schematic of an automotive vision system having a busgateway within the path of image raw data between camera and head unit,with the camera control back channel established via the gateways (whichmay comprise a bidirectional channel on one or both bus systems), andwith the head unit or other display device doing the graphic processing,and with the vision software or vision applications utilizing theincoming data for further (high level) processing, which may run ondesignated image processing hardware, and with the head unit carryingcommutation hardware for communicating with the camera, in accordancewith the present invention;

FIG. 29A is a schematic of an automotive vision system of the presentinvention, with a graphic engine (or the image processing) running as an‘app’ (application) or (evtl. additional) software on the headunit/display device's main control or on a graphic hardware, and withthe head unit connected to a phone or communication device;

FIG. 29B is a schematic of an automotive vision system of the presentinvention, with a graphic engine (or the image processing) running as an‘app’ (application) or (evtl. additional) software on the headunit/display device's main control or on a graphic hardware, with thehead unit connected to a phone or communication device and with awireless camera connected to the head unit via the connection to thephone or communication device and via a wireless communication hardwareat or to the phone or communication device;

FIG. 29C is a schematic of an automotive vision system of the presentinvention, with a graphic engine (or the image processing) running onthe head unit/display device's main control or on a graphic hardware,having at least one automotive camera and with a wireless cameraconnected to the head unit connected via wireless communication to awireless camera without control loop;

FIG. 29D is a schematic of an automotive vision system of the presentinvention, with a graphic engine (or the image processing) running onthe head unit/display device's main control or on a graphic hardware,having at least one automotive camera and with a wireless cameraconnected to the head unit connected via wireless communication to awireless camera with control loop;

FIG. 29E is a schematic of an automotive vision system of the presentinvention consecutive to FIG. 11, having several vehicle inherentautomotive cameras connected to a control device and with a wirelesscamera connected additionally via wireless communication hardware on thecontrol device to a wireless camera with control loop;

FIG. 29F is a schematic of an automotive vision system of the presentinvention, with a graphic engine (or the image processing) running onthe head unit/display device's main control or on a graphic hardware,having at least one automotive camera and with an external vehicleinherent wireless device connected to the head unit on which wirelesscamera with control loop is connected;

FIG. 29G is a schematic of an automotive vision or driver assistantsystem of the present invention, with a graphic engine (or the imageprocessing) running on the head unit/display device's main control or ona graphic hardware, having several automotive cameras connected and withwireless camera connected to the head unit;

FIG. 29H is a schematic of an automotive vision or driver assistantsystem of the present invention consecutive to FIG. 29G, with a mobilephone or communication device attached additionally;

FIG. 29I is a schematic of an automotive vision system of the presentinvention consecutive to FIG. 29B, with a first graphic engine (or theimage processing) running as an ‘app’ (application) or (evtl.additional) software on the head unit/display device's main control oron a graphic hardware, and with the head unit connected to a phone orcommunication device and with a wireless camera connected to the headunit via the connection to the phone or communication device and via awireless communication hardware at the phone or communication device andwith a second graphic engine (or the image processing) running as an‘app’ (application) or (evtl. additional) software on the mobile deviceexecuting a first part of the image processing while the graphic engineon the head unit is processing a second or alternative part of imageprocessing;

FIG. 29J is a schematic of an automotive vision system of the presentinvention consecutive to FIG. 29I, with a first graphic engine (or theimage processing) running as an ‘app’ (application) or (evtl.additional) software on the head unit/display device's main control oron a graphic hardware, and with the head unit connected to a phone orcommunication device and with a wireless camera connected to the headunit via the connection to the phone or communication device and via awireless communication hardware at the phone or communication device andwith a second graphic engine (or the image processing) running as an‘app’ (application) or (evtl. additional) software on the mobile deviceexecuting a first part of the image processing while the graphic engineon the head unit is processing a second or alternative part of imageprocessing, with more than one vehicle inherent cameras, other visiondevices, other sensors and other outputs and actuators attached to acommon bus or backbone for exchanging data and commands;

FIG. 29K is a schematic of an automotive vision system of the presentinvention consecutive to FIGS. 29I and 29J, with a graphic engine (orthe image processing) running as an ‘app’ (application) or (evtl.additional) software on the head unit/display device's main control oron a graphic hardware, and with a wireless camera connected to the headunit/display device via a wireless communication hardware at it and withmore than one vehicle inherent cameras, other vision devices, othersensors and other outputs and actuators attached to a common bus orbackbone for exchanging data and commands;

FIG. 29L is a schematic of an automotive vision system of the presentinvention consecutive to FIGS. 29I to 29J, with a graphic engine (or theimage processing) running as an ‘app’ (application) or (evtl.additional) software on the head unit/display device's main control oron a graphic hardware, and with a bus or backbone connected to the headunit and with several vehicle inherent cameras connected to the headunit via the back bone and with wireless camera connected via wirelesscommunication to a wireless communication device which is itself to thehead unit/display device via the backbone and with other vision devices,other sensors and other outputs and actuators attached to a common busor backbone for exchanging data and commands;

FIG. 29M is a schematic of an automotive vision system of the presentinvention consecutive to FIGS. 29I to 29L, with the image processing isrunning as an ‘app’ (application) or (evtl. additional) software orhardware in part on a communication device and in part on a dedicatedvehicle inherent image processing device (SVS) and/or in part on a headunit/display device, with the devices interconnected to each other bydata lines buses or back bones, and with a wireless camera attached viawireless connection to the communication device, and with the actuatorsmainly connected to the central display device, and with the centraldisplay device optionally displaying data coming from the wirelesscamera and coming from the vehicle inherent cameras;

FIG. 30 is a schematic of an automotive vision system of the presentinvention, with the head unit connected to at least one camera and aphone or communication device in accordance with FIG. 29A, and with theimage processing software running mainly on the head unit or otherdisplay device, wherein the image processing software becomes updatedfrom a remote device by transferring a data container containing anupdated version of software;

FIG. 31 is a schematic of an automotive vision system of the presentinvention, with the head unit or other display device connected to atleast one camera and a phone or communication device in accordance withFIG. 29A, and with a bidirectional data channel established between thecamera and the head unit, and wherein the image processing software isrunning mainly on the camera, but there may be parts of image processingsoftware that is running on the head unit or at a mobile ‘app’, with theimage processing software in the camera (and evtl. also these of thehead unit and/or communication device) becoming updated from a remotedevice or communication by transferring a data container containing anupdated version of software from the communication device to the camera,preferably over the head unit, and eventually directly via a common bus;

FIG. 32 is a schematic of an automotive vision system of the presentinvention, where the communication between head units or other displaydevices and mobile phones may use a layer based model, and such layerbased models may be used in the driver assistant and safety visionsystem's cameras;

FIG. 33 is a schematic of an automotive vision system of the presentinvention, wherein, similar to the general layer model from FIG. 32, avirtual communication layer comes into use, with the applicationcommunication layer communicating via virtual communication layer toother applications which may be located on other devices in a manner asif all applications are local, and showing examples of a communicationbetween applications on a mobile device and those at a head unitequipped with graphic processing hardware and graphic processingapplications, with the hardware link comprising data buses or channelssuch as CAN, LIN, Ethernet, Bluetooth, NFC (Near Field Communication),and/or the like;

FIG. 34 is a schematic of an automotive vision system of the presentinvention similar to that of FIG. 33, with a virtual communication layerextended over several devices, and with communication betweenapplications on a mobile device, a head unit and a camera, wherein theconnection may be capable to exchange image data, camera control,parameters from sensors and other devices, and driver assistant controlsand/or the like;

FIG. 35 is a schematic of an automotive vision system that uses datadecoders and/or line drivers in automotive vision camera systems, inaccordance with the present invention;

FIG. 36 is a schematic of an automotive vision system that usesLVDS/Ethernet data decoders for driving in automotive vision cameraimages in accordance with the present invention;

FIG. 37 is a schematic of an automotive vision system according to FIGS.9 and 10, with the LVDS/Ethernet (or other bus) driver chip sharing onedevice with the imager, and with the optics comprising a separatecomponent, in accordance with the present invention;

FIG. 38 is a schematic of an automotive vision system according to FIG.37, with the LVDS/Ethernet (or other bus) driver chip sharing one devicewith the imager, and with the optics incorporated to the substrate toprovide a single compact assembly in accordance with the presentinvention, and with the so called wafer level cameras incorporating thebus driver, with an optical assembly or lens assembly that may comprisea single optic or lens or multiple optics or lenses;

FIG. 39 is a schematic of a vehicle (10) viewed from top down with a oneaxis trailer (20) hooked on, wherein the trailer has its own rear camera(14 a′), which may be connected by wire or wireless to the vehicle'svision devices or a cell phone displaying the area of view (25) of thetrailer rear camera on a display or incorporating the image datainformation on a machine vision assist system in accordance with thepresent invention;

FIG. 40 is an exemplary view of a vehicle top view when an additionaltrailer camera (such as of the type shown in the set up of FIG. 39)sends images that are incorporated to the top view of the vehicle's maindisplay device;

FIG. 41 shows an overview of a smart camera suitable for use with avision system of the present invention;

FIG. 42-44 show view transformation and machine vision capabilities ofthe smart camera of FIG. 41;

FIG. 45 shows a summary of the various aspects and properties of theunique feature-point based object detection for use with a vision systemaccording the present invention;

FIG. 46 is a schematic showing a possible architecture solution when theinvention's processing algorithm are integrated to a smart vehiclecamera featuring different output bus interfaces; and

FIGS. 47-65 show various connections and system architecture and otheraspects and constructions and features of a smart camera and associatedaccessories and systems for use with a vision system according thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A driver assist system and/or vision system and/or object detectionsystem and/or alert system may operate to capture images exterior of thevehicle and process the captured image data to detect objects at or nearthe vehicle and in the predicted path of the vehicle, such as to assista driver of the vehicle in maneuvering the vehicle in a rearwarddirection. The object detection may utilize detection and analysis ofmoving vectors representative of objects detected in the field of viewof the vehicle camera, in order to determine which detected objects areobjects of interest to the driver of the vehicle, such as when thedriver of the vehicle undertakes a reversing maneuver.

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes one or more imaging sensors or cameras (such as a rearwardfacing imaging sensor or camera 14 a and/or a forwardly facing camera 14b at the front (or at the windshield) of the vehicle, and/or asidewardly/rearwardly facing camera 14 c, 14 b at the sides of thevehicle), which capture images exterior of the vehicle, with the camerashaving a lens for focusing images at or onto an imaging array or imagingplane of the camera (FIG. 1). The vision system 12 is operable toprocess image data captured by the cameras and may provide displayedimages at a display device 16 for viewing by the driver of the vehicle.Optionally, the vision system may process image data to detect objects,such as objects to the rear of the subject or equipped vehicle during areversing maneuver, or such as approaching or following vehicles orvehicles at a side lane adjacent to the subject or equipped vehicle orthe like. Optionally, the object detection may utilize detection andanalysis of moving vectors representative of objects detected in thefield of view of the vehicle camera, in order to determine whichdetected objects are objects of interest to the driver of the vehicle,such as when the driver of the vehicle undertakes a reversing maneuver(such as by utilizing aspects of the systems described in U.S.provisional applications, Ser. No. 61/605,409, filed Mar. 1, 2012; andSer. No. 61/563,965, filed Nov. 28, 2011, which are hereby incorporatedherein by reference in their entireties.

Vehicle vision systems typically use sophisticated image processingalgorithms. For example, either the algorithm runs on a hardware whichis integrated to a vision camera or the algorithm runs on an externalcontrol device collecting and processing the incoming camera raw data,and then sends a processed image data stream to a display device,cluster or head unit. Vehicle displays vehicle vision systems may beprovided at multi-media head units (such as by utilizing aspects of themirror assemblies described in U.S. Pat. No. 7,937,667, which is herebyincorporated herein by reference in its entirety), and may processseveral input sources, such that processing power is adaptedaccordingly. Optionally, a system may visualize and control the appsrunning on a mobile infotainment device or mobile phone which isconnected to or in communication with the head unit, either wirelessly(near field communication, BLUETOOTH®, and the like) or plugged in. Itis known to control the app interactively and depending on the drivingsituation.

The present invention provides a system that delocates the computingload from the camera or separate image control to the head unit orconjuncted or connected mobile device which typically come with highprocessing performance and are embedded into the display or head unitenvironment anyway. The system may utilize aspects of the systemsdescribed in U.S. Pat. No. 7,697,027 and/or U.S. patent application Ser.No. 13/377,673, filed Dec. 12, 2011, now U.S. Pat. No. 9,036,026, whichare hereby incorporated herein by reference in their entireties.

Vision system camera electronics are preferably developed to costefficient. Due to lower development costs, vision camera systems aretypically uniformed as much as possible to lower the number of variants.Also, the complexity is preferably reduced or limited or held as low aspossible. Every electronic part that is integrated into cameras producesheat and takes precious space. Simplifying and standardizing of camerasand their interfaces enables use of modern embedding methods, whichreduces system costs and enables the shrinking of the camera, whichprovide customer benefits.

A different or alternative approach is to eliminate dedicated visionsystem control devices by transferring its functions (intelligence)either into the (smart) cameras or by transferring its functions to theend devices as like the head unit, the (display) cluster, or otherdisplay device. By that also the IOs and controls (such as, for example,invasive assistant systems) may be transferred/placed on those devices.

For reducing the cabling costs, there is the trend to use busses/databackbones onto several sensing and actuator devices attached. Somesystems may be able to request data direct from another device on thebackbone without interaction of the main unit (head unit). This requiresat least bus interfaces on some or all nodes and maybe also processorsand may require a sophisticated state and priority handling across thedevices.

Vehicle development cycles are typically quite long compared to the rushchanging infotainment industry. Because of this, infotainment andvehicle driver assistant and safety systems (DAS) outdate quite rapidly.The appearance and functionality is often already a step behindafter-market systems when a vehicle inherent or vehicle-based systementers the market. To counter these circumstances, it would be desirableto vehicle manufacturers to keep (DAS) systems up to date.

The present invention provides an economized vehicle vision systemarchitecture that reduces the cost and complexity of the vehicle cameraand vision system.

(1) For lowering the number of variants of different car vision camerasystems, the inventive solution is to not implement the electronics in acamera or a separate image processing device and may transfer thegraphic engine to the head up unit. By avoiding this, the cameraelectronics can be reduced to the main functionalities for imagecapturing and own supply power control. Preferably, a simple datatransfer interface finds use to send the camera's (raw) data to the headunit (or other image giving/processing end device). On the head unit orother image giving/processing end device the graphic engine (or theimage processing) is processed. Other image giving/processing devicesmay comprise a mirror display within or outside of the passengercompartment, such as a mirror display at the top center region (such asa video mirror utilizing aspects of the displays described in U.S. Pat.Nos. 7,855,755 and/or 7,777,611, which are hereby incorporated herein byreference in their entireties). There it may be sufficient to placeimage processing tasks, and the hardware and software, when it comes tocomparably low sophisticated light balancing, distortion and stitchingtasks, may be fed by comparably low performance rear and side cameras.The early state of the art was to capture images and send it over datalines to a display, typically via an analog signal such as an NTSCsignal or the like. This was done without controlling or loopcontrolling the camera. The solution of the present invention alsoincorporates a control line running from the head unit to the camera. Aloop controlling of the camera and the graphic engine (or the imageprocessing) is realized by that. The controlling is necessary for lightand color balancing (such as described in PCT Application No.PCT/US2012/063520, filed Nov. 5, 2012, and published on May 16, 2013 asInternational Publication No. WO 2013/070539, which is herebyincorporated herein by reference in its entirety) and gain control.Also, the gamma correction may be controlled by the back channel. Othercontrol commands may by for sleep/wake up/idle, low high resolutionswitching, reduced area selection, compression control or triggering adata dump of intrinsic calibration data (such as described in U.S. Pat.No. 7,979,536, which is hereby incorporated herein by reference in itsentirety).

(2) Optionally, a continuative idea of the solution of the presentinvention is to realize the image data stream from the camera to thehead unit (or other end device) on a data encoder chip, preferably by aLow Voltage Differential Signal (LVDS) or Ethernet interface, preferablyusing a mono coaxial cable and Ethernet encoder chips (in the camera andin the head unit). The Data stream would be mono-directional (such as byutilizing aspects of U.S. provisional application 61/537,279, filed Sep.21, 2011, which is hereby incorporated herein by reference in itsentirety).

(3) Optionally, a further continuative idea is that the control line(HU-Cam) may be realized by a CAN/LIN/other car bus interface or just bya single signal line/wire.

(4) Optionally, and as an alternative to (3), above, the control signalmay be transferred via the data encoder chip, preferably byLVDS/Ethernet (data line from (2), above) so the interfaces and datastream are bi-directional.

(5) As solution for the graphic engine (or the image processing) (from(1), above) running on the head unit for a mono or multi camera system,processing the camera control, image dewarp, enhancing, adding overlays,DAS functionality and the like, the software could be running on ahardware added to the head unit.

(6) A continuative idea from (5), above, is to have the head unitconnected to a phone or communication device and to one or more cameras(see FIG. 29A).

(7) As a continuative idea to (6), above, the image processing softwareis running primarily on the head unit. The image processing softwarebecomes updated from a remote source or device by transferring a datacontainer containing an updated version of the image processingsoftware. The container is a frame which is made individually for eachvehicle communication environment. The content is preferably vehicletype and manufacturer independent, so the content may be substantiallystandardized to keep the variety low. Preferably, the communicationdevice enables an app to carry the container. The container may also betransported by an audio (voice) channel, SMS, MMS, DSRC, near fieldcommunication, via a pier to pier protocol and/or the like (see FIG.30). An alternative is to provide updates by a data medium, a removablecable or an OEM's service interface, either connected to a vehicle'sport, a head unit's port, a communication device's port or a visiondevice's port.

(8) In cases where the hardware set up allows bidirectionalcommunication between the head unit and the camera(s) and a phone orcommunication or between the communication device and the camera(s)directly, there is a continuative idea to (7), above, which may updatethe camera(s) image processing software by remote transferring datacontainers via the communication channels mentioned in (7) or by acommon bus, or by an OEM service interface within a data container, fromthe communication device over the head unit to the camera. This mayoccur at once or step by step. There may be a part of the imageprocessing software at the head unit, and this may be updated as wellwith the same or a different data container, at the same or another time(see FIG. 31).

(9) An addition solution to (7) and (8), above, is to have data securityprotocols in place making sure no data in the head unit, camera(s) orcommunication device or conjuncted system's software becomes overwrittenor corrupted by wrong versions or by draft versions, or unauthorizedversions, or by incomplete data sets or incompatible data sets, or bypirated versions or data sets, or virulent data sets, and/or the like.This task may also be managed by the data container's functionality.

(10a) The data of (9) preferably have been authorized by an applicationprovider or distributor, the camera or image system software provider ordistributer or the according OEM in compliance to the local or worldwidelegal or OEM's safety standards and testing procedures for safetyrelevant software and non-safety relevant software, whichever may applyfor the particular application.

(10b) As an additional or alternative solution, the image processing maybe done in part on the mobile device or cellular phone or smart phoneand in part on the head unit or other display device and/or in part in adedicated image processing unit (see FIG. 29M).

As an additional aspect to the inventive solution, one of the systemsdescribed above may supplement or substitute the vehicle inherent rearcamera's image input by an attached (aftermarket) non vehicle inherentcamera image. An exemplary use case is shown in FIG. 39, where a trailercamera 14 a′, which is attached to the rear of a trailer 20, and whichcamera's image is sent to the vehicle vision system or a non-vehicledisplay and/or communication device (such as a cell phone's display orthe like) via wireless communication (such as, for example, via aBluetooth communication or via a WLAN or NTSC communication or thelike). The architecture may be similar to those shown in FIGS. 29B, 29Eand 29G-M. In the cases where the vehicle vision system receives thewireless camera image, the vision system is displaying the trailercamera's image as well. The system may switch to use or display theimage of the vehicle rear camera 14 a, such as for (machine vision)trailer (side) blind spot detection and trailer angle detection. Thearea where usually the rear camera is placed within top view (mode) maybe filled with the image from the wireless camera so as to be seen inthe display (FIG. 40). Since the wireless trailer camera 14 a′ may notbe calibrated and some area between the vehicle rear camera 14 a and thetrailer camera 14 a′ may not be captured by any camera there may bevoids or black bars in the top view images between the areas encompassedby the vehicle inherent cameras and the area encompassed by trailercamera. The trailer camera may be acquired with the trailer, or may bepurchased independently (aftermarket) and then attached to the traileror may be an optional delivery content from the OEM distributer.

(11) Optionally, the system may use layer based models in thecommunication architecture between head units, actuators, sensors,communication devices, image processing and/or driver assistant devices,and mobile phones, and may use such layer based models in the driverassistant and safety vision system's cameras this might be new (see FIG.32). The devices may be interconnected by one or more data lines, bussesor backbones (see FIG. 29L), or may be interconnected over other devicesand or gateways (see FIG. 29M). The proposed model contains an“Application layer” in which the application's semaphores run, a “Middle(soft-)ware” (MW) which adapts to the operation system and provides thecommunication between the application components, an “Operation SystemAbstraction Layer” (OSAL) which provides the abstracted base services ofthe Operation system as like LINUX which has configurable tasks andevents and controls the communication behavior. Also, there is a “Runtime operation system” (RTOS) which is the operation system itself andthe “Board support package” (BSP) which adapts the operation system tothe specific hardware.

(12) The present invention provides for DAS systems to establish avirtual communication layer, which expands over several devices havingthe same layer structure, and is connected via the communicationhardware physically. The application communication layer communicatesvia virtual communication layer to other applications which are locatedon the same or other devices in a manner as if all applications would beon the local device (see FIGS. 33 and 34). Graphic capturing orprocessing applications might connect to each other by this. Thehardware link may comprise removable ports (mentioned in (7), above),data buses or channels, such as CAN, LIN, LVDS/Ethernet, Bluetooth, NFC(Near Field Communication), and the like.

(13) Using the layer model from (12), above, a continuative idea is totransport data containers (from 7) from one device to the other.

(14) The virtual communication layer may have security tasks as well fordoing that.

(15) Over these layers (from (12), above) there may become image data,camera control, parameters from sensors and other devices, driverassistant controls exchanged.

(16) Optionally, and as an alternative to (5), above, the graphic engine(or the image processing) of a mono or multi camera car vision systemcould be part of the head unit's software in full extend or partially,so a routine which preferably is served in real time may be provided.The processing performance and architecture of the head unit andperipheral interfaces has to be chosen or selected accordingly.

(17) Optionally, and as an alternative solution to (5) and (16), above,the head unit may be conjuncted to or connected to or in communicationwith a mobile device, such as, for example, a mobile infotainment deviceor a mobile phone. The graphic engine (or the image processing) may berunning as an ‘app’ (application) on the mobile device and processingcamera images which are not taken by the mobile phone's cameras butpreferably taken or captured by a fixed mounted car vision camera orcameras, a wireless or wired aftermarket camera or cameras or a USBcamera or cameras plugged into any USB port that is part of the carintegrated bus architecture or other car based plug in bus.

(18) Optionally, and as an additional feature to (17), above, the appused in the mobile device is certified by according certification boardsof governmental organs or (mobile device) companies and/or OEMs.

(19) Optionally, and as an additional feature to (17) or (18), above,the vision system's additional functions (value added) may be providedseparate to the camera with the software installed on the head unit oron the app conjuncted to the head unit or from a cloud server or thelike.

(20) Optionally, and as an additional solution to (19), above, thesoftware (SW) or app may additionally compute additional parameters, andmay consider or take into account various parameters, such as thesteering angle of the vehicle or the like. This might be used forprocessing graphical steering aid overlays to a rear cameras displayedimages while backing up the vehicle or for other driver assistantsystems overlays.

(21) Optionally, and as an additional solution to (19) or (20), above,the SW (app) may additionally compute or process more than one camera,and also other driver assistant system sources like ultrasound sensors,Radar sensors, infrared or visual cameras, Lidar or Laser sensors.

(22) Optionally, and as an additional solution to (19) to (21), above,the app may not just visualize driver assistant functions, but may alsocontrol driving interventions or active warnings, such as haptic alertsor the like, such as, for example, steering wheel vibrations or footpedal vibrations or the like.

(23) Optionally, and as an additional solution to (19) to (22), above,the app may be adaptable for being updated, debugged, licensed, remotecontrolled, purchased, sold on ‘app-stores’ or the like, leased, timelimited tried, reimbursed when given back, transferred, and/or the like.Also, the camera or vision system calibration may be running on the appor as an app.

(24) Optionally, and as a consecutive solution to (4), above, theLVDS/Ethernet (or other bus) driver chip may share one device with theimager. Thus, the present invention may provide a nearly monolyticassembly.

(25) Optionally, and as a consecutive solution to (24), above, andinstead of having a lens holder as a separate component, the optics ofthe camera or imager may be incorporated to the compact, monolyticdesign of the imager device. The lens assembly or optics may compriseone lens or lens optic or optical element, or a plurality of lenses orlens optics, such as many small ones. So called wafer level cameras arestate of the art in automotive applications but also incorporating thebus driver is new to such automotive camera applications.

(26) Optionally, any kind of automotive camera including wafer levelcameras from (25) above may comprise a tunable liquid (micro-) lenscapable to adapt the focal length of the lens. Optionally, the viewingdirection may also be tunable by a microelectronic mechanical (MEM) orother mechanical element, either by turning the camera or a mirror thatthe camera has in its optical path. Such an assembly may be capable tofocus on specific details or objects of interest within the field ofview to enhance the performance of the object detection of the driverassistant system. Instead of using wide angle cameras (such as like fisheye lens cameras and the like), a focus tunable camera with a generallylonger focus band may be used. The full area may be detectable by‘scanning’ through all of the extensions of the mechanics' displacementangles. By that the same area as that provided by a fisheye lens may becovered but with the ability to perceive the area much better andevenly. The focus point to objects in the area or field of view may be adegree to determine or at least to estimate the distance between thefocused object and the camera's imager (as like the eye of a chameleondoes). For example, an exterior viewing camera with a relatively narrowfield of view (for example, a horizontal field of view of about 40degrees) can be mechanically moved rapidly so that its field of viewscans or senses or rasters across a wider field of view (for example, ahorizontal field of view of about 200 degrees). If this mechanicalmotion of the principal viewing direction of the subject camera isexecuted fast enough (for example, if an entire scan can be executed inabout 30 ms or lower, video captured by the rastering camera, asdisplayed to the driver on an in-cabin video screen, will be usable tothat driver as the rate of movement of the rastering camera exceeds therate at which the eye perceives.

Also, if the focus of the camera can be varied, this can be used toestimate or determine distance to an object, such as discussed below.For illustrative purposes, assume that the focus-variable lens has afive meter focal point/length and assume that a person is standing 10meters away from the subject lens/camera. The image of the person ascaptured by that camera will be out of focus. Then, as the focalpoint/length is progressively increased to six meters, then seven metersand then eight meters and then nine meters, the person ten meters awayfrom the camera remains out of focus and comes into focus only when thevariable-focus lens is set to have an about ten meter focalpoint/length. Thus, the distance from the camera can be determined orestimated based on varying the focal point/length of the lens anddetermining when the imaged object comes into focus in the capturedimages.

The camera or sensor may comprise any suitable camera or sensor.Optionally, the camera may comprise a “smart camera” that includes theimaging sensor array and associated circuitry and image processingcircuitry and electrical connectors and the like as part of a cameramodule, such as by utilizing aspects of the vision systems describedabove and shown in FIGS. 41-65.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ultrasonic sensors or thelike. The imaging sensor or camera may capture image data for imageprocessing and may comprise any suitable camera or sensing device, suchas, for example, an array of a plurality of photosensor elementsarranged in at least 640 columns and 480 rows (at least a 640×480imaging array), with a respective lens focusing images onto respectiveportions of the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. The logic and control circuit of the imaging sensor mayfunction in any known manner, and the image processing and algorithmicprocessing may comprise any suitable means for processing the imagesand/or image data. For example, the vision system and/or processingand/or camera and/or circuitry may utilize aspects described in U.S.Pat. Nos. 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935;6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229;7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287;5,929,786 and/or 5,786,772, and/or PCT Application No.PCT/US2010/047256, filed Aug. 31, 2010 and published Mar. 10, 2011 asInternational Publication No. WO 2011/028686 and/or InternationalPublication No. WO 2010/099416, published Sep. 2, 2010, and/or PCTApplication No. PCT/US10/25545, filed Feb. 26, 2010 and published Sep.2, 2010 as International Publication No. WO 2010/099416, and/or PCTApplication No. PCT/US2012/048800, filed Jul. 30, 2012, and published onFeb. 7, 2013 as International Publication No. WO 2013019707, and/or PCTApplication No. PCT/US2012/048110, filed Jul. 25, 2012, and publishedJan. 31, 2013 as International Publication No. WO 2013016409, and/or PCTApplication No. PCT/CA2012/000378, filed Apr. 25, 2012, and published onNov. 1, 2012 as International Publication No. WO 2012145822, and/or PCTApplication No. PCT/US2012/056014, filed Sep. 19, 2012, and published onMar. 28, 2013 as International Publication No. WO 2013043661, and/or PCTApplication No. PCT/US12/57007, filed Sep. 25, 2012, and published onApr. 4, 2013 as International Publication No. WO 2013/048994, and/or PCTApplication No. PCT/US2012/061548, filed Oct. 24, 2012, and publishedMay 2, 2013 as International Publication No. WO 2013063014, and/or PCTApplication No. PCT/US2012/062906, filed Nov. 1, 2012, and published May10, 2013 as International Publication No. WO 2013067083, and/or PCTApplication No. PCT/US2012/063520, filed Nov. 5, 2012, and published May16, 2013 as International Publication No. WO 2013070539, and/or PCTApplication No. PCT/US2012/064980, filed Nov. 14, 2012, and publishedMay 23, 2013 as International Publication No. WO 2013074604, and/or U.S.patent application Ser. No. 13/660,306, filed Oct. 25, 2012, now U.S.Pat. No. 9,146,898; Ser. No. 13/653,577, filed Oct. 17, 2012, now U.S.Pat. No. 9,174,574; and/or Ser. No. 13/534,657, filed Jun. 27, 2012, andpublished on Jan. 3, 2013 as U.S. Patent Publication No.US-2013-0002873, and/or U.S. provisional applications, Ser. No.61/710,924, filed Oct. 8, 2012; Ser. No. 61/696,416, filed Sep. 4, 2012;Ser. No. 61/682,995, filed Aug. 14, 2012; Ser. No. 61/682,486, filedAug. 13, 2012; Ser. No. 61/680,883, filed Aug. 8, 2012; Ser. No.61/678,375, filed Aug. 1, 2012; Ser. No. 61/676,405, filed Jul. 27,2012; Ser. No. 61/666,146, filed Jun. 29, 2012; Ser. No. 61/653,665,filed May 31, 2012; Ser. No. 61/653,664, filed May 31, 2012; Ser. No.61/648,744, filed May 18, 2012; Ser. No. 61/624,507, filed Apr. 16,2012; Ser. No. 61/616,126, filed Mar. 27, 2012; Ser. No. 61/615,410,filed Mar. 26, 2012; Ser. No. 61/613,651, filed Mar. 21, 2012; Ser. No.61/607,229, filed Mar. 6, 2012; Ser. No. 61/602,878, filed Feb. 24,2012; Ser. No. 61/602,876, filed Feb. 24, 2012; Ser. No. 61/600,205,filed Feb. 17, 2012; Ser. No. 61/588,833, filed Jan. 20, 2012; Ser. No.61/583,381, filed Jan. 5, 2012; Ser. No. 61/570,017, filed Dec. 13,2011; Ser. No. 61/568,791, filed Dec. 9, 2011; Ser. No. 61/567,446,filed Dec. 6, 2011; and/or Ser. No. 61/567,150, filed Dec. 6, 2011,which are all hereby incorporated herein by reference in theirentireties. The system may communicate with other communication systemsvia any suitable means, such as by utilizing aspects of the systemsdescribed in PCT Application No. PCT/US10/038477, filed Jun. 14, 2010,and/or U.S. patent application Ser. No. 13/202,005, filed Aug. 17, 2011,now U.S. Pat. No. 9,126,525, which are hereby incorporated herein byreference in their entireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras and vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,123,168;7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176; 6,313,454 and6,824,281, and/or International Publication No. WO 2010/099416,published Sep. 2, 2010, and/or PCT Application No. PCT/US10/47256, filedAug. 31, 2010 and published Mar. 10, 2011 as International PublicationNo. WO 2011/028686, and/or U.S. patent application Ser. No. 12/508,840,filed Jul. 24, 2009, and published Jan. 28, 2010 as U.S. Pat.Publication No. US 2010-0020170, and/or PCT Application No.PCT/US2012/048110, filed Jul. 25, 2012, and published Jan. 31, 2013 asInternational Publication No. WO 2013016409, and/or U.S. patentapplication Ser. No. 13/534,657, filed Jun. 27, 2012, and published Jan.3, 2013 as U.S. Patent Publication No. US-2013-0002873, which are allhereby incorporated herein by reference in their entireties. The cameraor cameras may comprise any suitable cameras or imaging sensors orcamera modules, and may utilize aspects of the cameras or sensorsdescribed in U.S. patent application Ser. No. 12/091,359, filed Apr. 24,2008 and published Oct. 1, 2009 as U.S. Publication No. US-2009-0244361,and/or Ser. No. 13/260,400, filed Sep. 26, 2011, now U.S. Pat. No.8,542,451, and/or U.S. Pat. Nos. 7,965,336 and/or 7,480,149, which arehereby incorporated herein by reference in their entireties. The imagingarray sensor may comprise any suitable sensor, and may utilize variousimaging sensors or imaging array sensors or cameras or the like, such asa CMOS imaging array sensor, a CCD sensor or other sensors or the like,such as the types described in U.S. Pat. Nos. 5,550,677; 5,670,935;5,760,962; 5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610;6,590,719; 6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176;6,559,435; 6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978;7,339,149; 7,038,577; 7,004,606 and/or 7,720,580, and/or U.S. patentapplication Ser. No. 10/534,632, filed May 11, 2005, now U.S. Pat. No.7,965,336; and/or PCT Application No. PCT/US2008/076022, filed Sep. 11,2008 and published Mar. 19, 2009 as International Publication No. WO2009/036176, and/or PCT Application No. PCT/US2008/078700, filed Oct. 3,2008 and published Apr. 9, 2009 as International Publication No. WO2009/046268, which are all hereby incorporated herein by reference intheir entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149 and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176 and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978 and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,720,580; 7,038,577; 5,929,786and/or 5,786,772, and/or U.S. pat. application Ser. No. 11/239,980,filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496, and/or U.S.provisional applications, Ser. No. 60/628,709, filed Nov. 17, 2004; Ser.No. 60/614,644, filed Sep. 30, 2004; Ser. No. 60/618,686, filed Oct. 14,2004; Ser. No. 60/638,687, filed Dec. 23, 2004, which are herebyincorporated herein by reference in their entireties, a video device forinternal cabin surveillance and/or video telephone function, such asdisclosed in U.S. Pat. Nos. 5,760,962; 5,877,897; 6,690,268 and/or7,370,983, and/or U.S. patent application Ser. No. 10/538,724, filedJun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US-2006-0050018, which are hereby incorporated herein by reference intheir entireties, a traffic sign recognition system, a system fordetermining a distance to a leading or trailing vehicle or object, suchas a system utilizing the principles disclosed in U.S. Pat. Nos.6,396,397 and/or 7,123,168, which are hereby incorporated herein byreference in their entireties, and/or the like.

Optionally, the circuit board or chip may include circuitry for theimaging array sensor and or other electronic accessories or features,such as by utilizing compass-on-a-chip or EC driver-on-a-chip technologyand aspects such as described in U.S. Pat. No. 7,255,451 and/or U.S.Pat. No. 7,480,149; and/or U.S. patent application Ser. No. 11/226,628,filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008, and/or Ser. No. 12/578,732, filed Oct. 14, 2009, nowU.S. Pat. No. 9,487,144, which are hereby incorporated herein byreference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. patentapplication Ser. No. 13/333,337, filed Dec. 21, 2011, now U.S. Pat. No.9,264,672, which are hereby incorporated herein by reference in theirentireties. The video mirror display may comprise any suitable devicesand systems and optionally may utilize aspects of the compass displaysystems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341;7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305;5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727;5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851,and/or European patent application, published Oct. 11, 2000 underPublication No. EP 0 1043566, and/or U.S. patent application Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, which are all hereby incorporatedherein by reference in their entireties. Optionally, the video mirrordisplay screen or device may be operable to display images captured by arearward viewing camera of the vehicle during a reversing maneuver ofthe vehicle (such as responsive to the vehicle gear actuator beingplaced in a reverse gear position or the like) to assist the driver inbacking up the vehicle, and optionally may be operable to display thecompass heading or directional heading character or icon when thevehicle is not undertaking a reversing maneuver, such as when thevehicle is being driven in a forward direction along a road (such as byutilizing aspects of the display system described in PCT Application No.PCT/US2011/056295, filed Oct. 14, 2011 and published Apr. 19, 2012 asInternational Publication No. WO 2012/051500, which is herebyincorporated herein by reference in its entirety).

Optionally, the vision system (utilizing the forward facing camera and arearward facing camera and other cameras disposed at the vehicle withexterior fields of view) may be part of or may provide a display of atop-down view or birds-eye view system of the vehicle or a surround viewat the vehicle, such as by utilizing aspects of the vision systemsdescribed in PCT Application No. PCT/US10/25545, filed Feb. 26, 2010 andpublished on Sep. 2, 2010 as International Publication No. WO2010/099416, and/or PCT Application No. PCT/US10/47256, filed Aug. 31,2010 and published Mar. 10, 2011 as International Publication No. WO2011/028686, and/or PCT Application No. PCT/US2011/062834, filed Dec. 1,2011 and published Jun. 7, 2012 as International Publication No. WO2012/075250, and/or PCT Application No. PCT/US2012/048993, filed Jul.31, 2012, and published Feb. 7, 2013 as International Publication No. WO2013019795, and/or PCT Application No. PCT/US11/62755, filed Dec. 1,2011 and published Jun. 7, 2012 as International Publication No. WO2012-075250, and/or PCT Application No. PCT/CA2012/000378, filed Apr.25, 2012, and published Nov. 1, 2012 as International Publication No. WO2012145822, and/or U.S. patent application Ser. No. 13/333,337, filedDec. 21, 2011, now U.S. Pat. No. 9,264,672, and/or U.S. provisionalapplications, Ser. No. 61/615,410, filed Mar. 26, 2012; Ser. No.61/588,833, filed Jan. 20, 2012; Ser. No. 61/570,017, filed Dec. 13,2011; and/or Ser. No. 61/568,791, filed Dec. 9, 2011, which are herebyincorporated herein by reference in their entireties.

Optionally, a video mirror display may be disposed rearward of andbehind the reflective element assembly and may comprise a display suchas the types disclosed in U.S. Pat. Nos. 5,530,240; 6,329,925;7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,370,983; 7,338,177;7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187 and/or6,690,268, and/or in U.S. patent application Ser. No. 12/091,525, filedApr. 25, 2008, now U.S. Pat. No. 7,855,755; Ser. No. 11/226,628, filedSep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008; and/or Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare all hereby incorporated herein by reference in their entireties. Thedisplay is viewable through the reflective element when the display isactivated to display information. The display element may be any type ofdisplay element, such as a vacuum fluorescent (VF) display element, alight emitting diode (LED) display element, such as an organic lightemitting diode (OLED) or an inorganic light emitting diode, anelectroluminescent (EL) display element, a liquid crystal display (LCD)element, a video screen display element or backlit thin film transistor(TFT) display element or the like, and may be operable to displayvarious information (as discrete characters, icons or the like, or in amulti-pixel manner) to the driver of the vehicle, such as passenger sideinflatable restraint (PSIR) information, tire pressure status, and/orthe like. The mirror assembly and/or display may utilize aspectsdescribed in U.S. Pat. Nos. 7,184,190; 7,255,451; 7,446,924 and/or7,338,177, which are all hereby incorporated herein by reference intheir entireties. The thicknesses and materials of the coatings on thesubstrates of the reflective element may be selected to provide adesired color or tint to the mirror reflective element, such as a bluecolored reflector, such as is known in the art and such as described inU.S. Pat. Nos. 5,910,854; 6,420,036 and/or 7,274,501, which are herebyincorporated herein by reference in their entireties.

Optionally, the display or displays and any associated user inputs maybe associated with various accessories or systems, such as, for example,a tire pressure monitoring system or a passenger air bag status or agarage door opening system or a telematics system or any other accessoryor system of the mirror assembly or of the vehicle or of an accessorymodule or console of the vehicle, such as an accessory module or consoleof the types described in U.S. Pat. Nos. 7,289,037; 6,877,888;6,824,281; 6,690,268; 6,672,744; 6,386,742 and 6,124,886, and/or U.S.patent application Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare hereby incorporated herein by reference in their entireties.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

The invention claimed is:
 1. A vehicular vision system, said vehicularvision system comprising: a plurality of cameras disposed at a vehicleequipped with said vehicular vision system, each viewing exterior of theequipped vehicle; said plurality of cameras comprising a forward viewingcamera disposed at and viewing at least forward through a windshield ofthe equipped vehicle, said forward viewing camera comprising an imagingsensor having a plurality of photosensor elements arranged in an arrayof multiple columns and multiple rows, said forward viewing cameraoperable to capture image data; said plurality of cameras comprising arear camera disposed at a rear portion of the equipped vehicle, saidrear camera comprising an imaging sensor having a plurality ofphotosensor elements arranged in an array of multiple columns andmultiple rows, said rear camera viewing at least rearward of theequipped vehicle and operable to capture image data; said forwardviewing camera and said rear camera each comprising at least oneselected from the group consisting of (i) a respective Low VoltageDifferential Signaling (LVDS) chip and (ii) a respective Ethernet chip;a control disposed at the equipped vehicle, said control comprising animage processor, wherein said forward viewing camera communicates withsaid control via a first image data transfer channel, and wherein saidrear camera communicates with said control via a second image datatransfer channel; wherein image data captured by said forward viewingcamera is communicated by the first image data transfer channel via afirst coaxial cable; wherein image data captured by said rear camera iscommunicated by the second image data transfer channel via a secondcoaxial cable; wherein image data captured by said forward viewingcamera and carried to said control by said first coaxial cable isprocessed at said image processor of said control for object detection;wherein said control generates an output provided to a video displaydevice of the equipped vehicle, said video display device comprising avideo display screen viewable by a driver of the equipped vehicle; andwherein said video display screen displays video images of an arearearward the equipped vehicle derived from image data captured by saidrear camera and carried to said control by said second coaxial cable. 2.The vehicular vision system of claim 1, wherein image data captured byat least said rear camera of said plurality of cameras is processed atsaid image processor of said control to detect an object presentrearward of the equipped vehicle.
 3. The vehicular vision system ofclaim 2, wherein, responsive to said processing of captured image dataat said image processor of said control, said vehicular vision systemdetermines movement vectors, and wherein, responsive to determination ofmovement vectors, said vehicular vision system determines that thedetected object is an object of interest rearward of the equippedvehicle.
 4. The vehicular vision system of claim 2, wherein, responsiveat least in part to determining, via processing of captured image dataat said image processor of said control, that the detected object is ahazardous object rearward of the equipped vehicle, an alert is providedto the driver of the equipped vehicle.
 5. The vehicular vision system ofclaim 1, wherein said control is operable to process input from at leastone selected from the group consisting of an ultrasound sensor, a radarsensor, an infrared sensor and a Lidar sensor.
 6. The vehicular visionsystem of claim 1, wherein said control receives input from at least onenon-permanently mounted device.
 7. The vehicular vision system of claim6, wherein at least one selected from the group consisting of (i) saidnon-permanently mounted device is plugged into a port attached to a busarchitecture of the equipped vehicle, (ii) said non-permanently mounteddevice is wirelessly connected to a bus architecture of the equippedvehicle, (iii) said non-permanently mounted device comprises a mobilephone device and (iv) said non-permanently mounted device comprises acamera.
 8. The vehicular vision system of claim 6, wherein saidnon-permanently mounted device comprises a camera that is wirelesslylinked to the equipped vehicle.
 9. The vehicular vision system of claim1, wherein each camera of said plurality of cameras comprises a waferlevel camera.
 10. The vehicular vision system of claim 1, wherein saidcontrol is disposed at a head unit of the equipped vehicle.
 11. Thevehicular vision system of claim 1, wherein said control comprises agraphic engine.
 12. The vehicular vision system of claim 1, wherein saidrear camera comprises a rear backup camera of the equipped vehicle. 13.The vehicular vision system of claim 1, wherein said plurality ofcameras comprises (i) a driver-side camera disposed at a driver side ofthe equipped vehicle, said driver-side camera comprising an imagingsensor having a plurality of photosensor elements arranged in an arrayof multiple columns and multiple rows, said driver-side camera viewingat least sideward of the driver side of the equipped vehicle andoperable to capture image data, and wherein said driver-side camera hasat least one selected from the group consisting of (i) an LVDS chip and(ii) an Ethernet chip, and wherein said driver-side camera connects withsaid control via a third coaxial cable, and wherein said third coaxialcable carries control data from said control to said driver-side cameraand carries image data captured by said driver-side camera from saiddriver-side camera to said control, and (ii) a passenger-side cameradisposed at a passenger side of the equipped vehicle, saidpassenger-side camera comprising an imaging sensor having a plurality ofphotosensor elements arranged in an array of multiple columns andmultiple rows, said passenger-side camera viewing at least sideward ofthe passenger side of the equipped vehicle and operable to capture imagedata, and wherein said passenger-side camera has at least one selectedfrom the group consisting of (i) an LVDS chip and (ii) an Ethernet chip,and wherein said passenger-side camera connects with said control via afourth coaxial cable, and wherein said fourth coaxial cable carriescontrol data from said control to said passenger-side camera and carriesimage data captured by said passenger-side camera from saidpassenger-side camera to said control.
 14. The vehicular vision systemof claim 13, wherein said driver-side camera disposed at the driver sideof the equipped vehicle is part of a driver-side exterior mirrorassembly of the equipped vehicle, and wherein said passenger-side cameradisposed at the passenger side of the equipped vehicle is part of apassenger-side exterior mirror assembly of the equipped vehicle.
 15. Thevehicular vision system of claim 13, wherein said video display screenis operable to display birds-eye view video images derived, at least inpart, from image data captured by (i) said rear camera, (ii) saiddriver-side camera and (iii) said passenger-side camera.
 16. Thevehicular vision system of claim 13, wherein said plurality of camerascomprises a front camera disposed at a front portion of the equippedvehicle, said front camera comprising an imaging sensor having aplurality of photosensor elements arranged in an array of multiplecolumns and multiple rows, said front camera viewing at least forward ofthe equipped vehicle and operable to capture image data, and whereinsaid front camera connects with said control via a fifth coaxial cable,and wherein said fifth coaxial cable carries control data from saidcontrol to said front camera and carries image data captured by saidfront camera from said front camera to said control.
 17. A vehicularvision system, said vehicular vision system comprising: a plurality ofcameras disposed at a vehicle equipped with said vehicular visionsystem, each viewing exterior of the equipped vehicle; said plurality ofcameras comprising a forward viewing camera disposed at and viewing atleast forward through a windshield of the equipped vehicle, said forwardviewing camera comprising an imaging sensor having a plurality ofphotosensor elements arranged in an array of multiple columns andmultiple rows, said forward viewing camera operable to capture imagedata; said plurality of cameras comprising a rear camera disposed at arear portion of the equipped vehicle, said rear camera comprising animaging sensor having a plurality of photosensor elements arranged in anarray of multiple columns and multiple rows, said rear camera viewing atleast rearward of the equipped vehicle and operable to capture imagedata, wherein said rear camera comprises a rear backup camera of theequipped vehicle; said forward viewing camera and said rear camera eachcomprising at least one selected from the group consisting of (i) arespective Low Voltage Differential Signaling (LVDS) chip and (ii) arespective Ethernet chip; a control disposed at the equipped vehicle,said control comprising an image processor, wherein said forward viewingcamera communicates with said control via a first image data transferchannel, and wherein said rear camera communicates with said control viaa second image data transfer channel; wherein image data captured bysaid forward viewing camera is communicated by the first image datatransfer channel via a first coaxial cable; wherein image data capturedby said rear camera is communicated by the second image data transferchannel via a second coaxial cable; wherein image data captured by saidforward viewing camera and carried to said control by said first coaxialcable is processed at said image processor of said control for objectdetection; wherein image data captured by at least said rear camera ofsaid plurality of cameras is processed at said image processor of saidcontrol to detect an object present rearward of the equipped vehicle;wherein said control is operable to process input from a radar sensor;wherein said control generates an output provided to a video displaydevice of the equipped vehicle, said video display device comprising avideo display screen viewable by a driver of the equipped vehicle; andwherein, during a reversing maneuver of the equipped vehicle, said videodisplay screen displays video images of an area rearward the equippedvehicle derived from image data captured by said rear camera and carriedto said control by said second coaxial cable.
 18. The vehicular visionsystem of claim 17, wherein said control determines, via processing ofcaptured image data at said image processor of said control, that thedetected object is a hazardous object rearward of the equipped vehicle.19. The vehicular vision system of claim 18, wherein, responsive atleast in part to determination that the detected object is a hazardousobject rearward of the equipped vehicle, an alert is provided to thedriver of the equipped vehicle.
 20. The vehicular vision system of claim17, wherein said control receives input from at least onenon-permanently mounted device, and wherein at least one selected fromthe group consisting of (i) said non-permanently mounted device isplugged into a port attached to a bus architecture of the equippedvehicle, (ii) said non-permanently mounted device is wirelesslyconnected to a bus architecture of the equipped vehicle, (iii) saidnon-permanently mounted device comprises a mobile phone device and (iv)said non-permanently mounted device comprises a camera.
 21. A vehicularvision system, said vehicular vision system comprising: a plurality ofcameras disposed at a vehicle equipped with said vehicular visionsystem, each viewing exterior of the equipped vehicle; said plurality ofcameras comprising a forward viewing camera disposed at and viewing atleast forward through a windshield of the equipped vehicle, said forwardviewing camera comprising an imaging sensor having a plurality ofphotosensor elements arranged in an array of multiple columns andmultiple rows, said forward viewing camera operable to capture imagedata; said plurality of cameras comprising a rear camera disposed at arear portion of the equipped vehicle, said rear camera comprising animaging sensor having a plurality of photosensor elements arranged in anarray of multiple columns and multiple rows, said rear camera viewing atleast rearward of the equipped vehicle and operable to capture imagedata; said plurality of cameras comprising a driver-side camera disposedat a driver side of the equipped vehicle, said driver-side cameracomprising an imaging sensor having a plurality of photosensor elementsarranged in an array of multiple columns and multiple rows, saiddriver-side camera viewing at least sideward of the driver side of theequipped vehicle and operable to capture image data; said plurality ofcameras comprising a passenger-side camera disposed at a passenger sideof the equipped vehicle, said passenger-side camera comprising animaging sensor having a plurality of photosensor elements arranged in anarray of multiple columns and multiple rows, said passenger-side cameraviewing at least sideward of the passenger side of the equipped vehicleand operable to capture image data; said plurality of cameras comprisinga front camera disposed at a front portion of the equipped vehicle, saidfront camera comprising an imaging sensor having a plurality ofphotosensor elements arranged in an array of multiple columns andmultiple rows, said front camera viewing at least forward of theequipped vehicle and operable to capture image data; each camera of saidplurality of cameras comprising at least one selected from the groupconsisting of (i) a respective Low Voltage Differential Signaling (LVDS)chip and (ii) a respective Ethernet chip; a control disposed at theequipped vehicle, said control comprising an image processor, whereinsaid forward viewing camera communicates with said control via a firstimage data transfer channel, and wherein said rear camera communicateswith said control via a second image data transfer channel, and whereinsaid driver-side camera communicates with said control via a third imagedata transfer channel, and wherein said passenger-side cameracommunicates with said control via a fourth image data transfer channel,and wherein said front camera communicates with said control via a fifthimage data transfer channel; wherein image data captured by said forwardviewing camera is communicated by the first image data transfer channelvia a first coaxial cable; wherein image data captured by said rearcamera is communicated by the second image data transfer channel via asecond coaxial cable; wherein image data captured by said driver-sidecamera is communicated by the third image data transfer channel via athird coaxial cable; wherein image data captured by said passenger-sidecamera is communicated by the fourth image data transfer channel via afourth coaxial cable; wherein image data captured by said front camerais communicated by the fifth image data transfer channel via a fifthcoaxial cable; wherein image data captured by said forward viewingcamera and carried to said control by said first coaxial cable isprocessed at said image processor of said control for object detection;wherein said control generates an output provided to a video displaydevice of the equipped vehicle, said video display device comprising avideo display screen viewable by a driver of the equipped vehicle; andwherein said video display screen displays birds-eye view video imagesderived, at least in part, from (i) image data captured by said rearcamera and carried to said control by said second coaxial cable, (ii)image data captured by said driver-side camera and carried to saidcontrol by said third coaxial cable, (iii) image data captured by saidpassenger-side camera and carried to said control by said fourth coaxialcable and (iv) image data captured by said front camera and carried tosaid control by said fifth coaxial cable.
 22. The vehicular visionsystem of claim 21, wherein image data captured by at least said rearcamera of said plurality of cameras is processed at said image processorof said control to detect an object present rearward of the equippedvehicle.
 23. The vehicular vision system of claim 22, wherein,responsive at least in part to determining, via processing of capturedimage data at said image processor of said control, that the detectedobject comprises a hazardous object rearward of the equipped vehicle, analert is provided to the driver of the equipped vehicle.
 24. Thevehicular vision system of claim 21, wherein said driver-side cameradisposed at the driver side of the equipped vehicle is part of adriver-side exterior mirror assembly of the equipped vehicle, andwherein said passenger-side camera disposed at the passenger side of theequipped vehicle is part of a passenger-side exterior mirror assembly ofthe equipped vehicle.